Minimally invasive surgery is performed using elongated instruments inserted into the patient's body through small ports. The main visualization method during these procedures is an endoscope. In robotic guided minimally invasive surgery, one or more of the instruments is held and controlled by a robotic device, particularly the endoscope.
Specifically, the small ports that are placed on the patient's body are the only incision points through which the instruments and endoscope may pass to access the inside of the patient. As such, the instruments can rotate around these fulcrum points, but they cannot impose translational forces on the ports, as this would cause injury and harm to the patient. This is especially important for robotic guided surgery, as the robot may potentially exert large forces.
As such, some known robots implement what is known as a remote center of motion (RCM) at the fulcrum point by enforcing that only rotation may be performed at the port and all translational forces at the port are eliminated. This can be achieved by implementing a mechanical design which has a remote center of motion at a specific location in space, and then aligning that point in space with the port.
Once the endoscope is inserted into the patient's body and an appropriate location of RCM is selected, the robot can be controlled from endoscope images. To close that control loop, a mathematical transformation between image coordinates and robot joint space has to be established (referred to in the art as Image Jacobian). The entire process is referred to in the art as the system calibration and requires various steps such as camera and robot calibration. Furthermore, to provide full calibration, depth between the camera and the organ/object under consideration needs to be measured and such measurements are typically either from images or using special sensors.
Intraoperative X-ray imaging may be used to supplement endoscopy during the procedure. For example, for minimally invasive coronary bypass surgery, intraoperative X-ray is used to identify arterial sclerosis or to confirm revascularization. As this imaging modality uses ionizing radiation, it is beneficial for the patient and the operation room staff that the number of X-ray images acquired during the procedure is reduced.
In order to improve utility of both imaging modalities, the system has to allow target cross-identification between the modalities. For example, if an arterial stenosis is not visible in an endoscope image and is visible in an X-ray image, a method to integrate these images and depict the stenosis in the endoscope image would greatly improve the workflow and reduce both operation time and reduce likelihood of complications. Further, control of the endoscope from X-ray images would lead to better visualization and detection of targets outside of endoscope's field-of-view.
In order to perform all these integrations, a calibration of endoscope as well as calibration of the robot has to be performed. The process to calibrate a robot and endoscope in the operating theater is tied to various issues.
One issue is camera calibration is an additional step that needs to be performed prior to the surgery by a trained person. This process is time-consuming and error prone.
A second issue is, if the user changes some of the camera parameters during surgery (e.g., zoom), then the endoscope has to be removed from the patient and recalibrated which might not be acceptable as it would interrupt the workflow.
A third issue is, if the user moves endoscope relative to the robot (e.g., inserts deeper into the body to perform mechanical zoom with a common class of endoscope that do not have optical zoom), then the system has to be either recalibrated or the robot must utilize additional sensors.
A fourth issue is calibration may require depth information, which might be accessible using depth finding algorithms (e.g., shape from motion or similar), but those kind of measurements are usually noisy, which might cause instability.
These issues may reduce usability of the system and disturb the workflow which will lead to an increase of procedure cost and lower acceptance rate.